Amendments to the Drawings: 

The attached sheet of drawings includes changes to Figure 1-6. 
This sheet replaces the original Figures 1-6. Figure 6 has been designated as — 
Prior Art—. Formals drawings are submitted herewith under separate Letter to the 
Draftsperson which incorporate the changes required by the Examiner. Approval 
by the Examiner is respectfully requested. 

Attachment: Replacement Figures 1 -6 

Annoted Sheet Showing Changes to Figure 6 
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REMARKS 

Claims 1-14 were pending in the application. Claims 1, 4-9, 1 1-14 
stand rejected. Claims 2-3 and 10 stand objected to. Claim 2 was amended. 
Claims 15-20 were added. Claims 1-20 remain in the application. 

The drawings were objected to under 37 CFR 1 .83(a) and have 
been corrected. 

Claims 1, 4,7-9 and 13-14 stand rejected under 35 U.S.C. 103(a) as 
being xmpatentable over a combination of Szeliski et al. (US 6,044,1 81) and 
Huber (automatic 3D modeling using range images obtained from imknown 
viewpoints — IDS; hereafter "Huber"). The rejection states in relation to Szeliski: 

"With regard to claim 1, which is representative of claim 9, 
Szeliski discloses a method for deriving a three-dimensional model of a 
scene from a plurality of images of the scene (figures 1-3), said method 
comprising: 

"(a) generating a plurality of three-dimensional panoramic 
images of a scene (figures 2-3; column 4, lines 45-65); 

"(b) determining transformations that align the plurality of 
three-dimensional panoramic images (figures 4-5, also see colimm 13 line 
25 to colunrm 14 line 45); 

"(c) integrating spatial information from the plurality of 
three-dimensional panoramic images to form a spatial three-dimensional 
model of the scene (figures 1 and 2B; environment/ texture map memory 
270 stores a 3-D model); 

"(d) integrating intensity (column 17, lines 12-25) [It is 
believed colxmm 27, lines 12-25 was intended here] and texture 
information (figure 2B) from the plurality of three-dimensional panoramic 
images (figure 3) onto the spatial three-dimensional model to form a three- 
dimensional model of the scene (270 in figure 2B) containing both spatial 
and intensity information (figures 1 1-12)." 

Szeliski does not support the rejection. The rejection argues that 
Szeliski discloses steps (a)-(d) of the method of Claim 1, but, in relation to step 
(a), cites portions of Szeliski directed to the entirety of the method of Szeliski. 
The rejection states: 


-9- 


"(a) generating a plurality of three-dimensional panoramic 
images of a scene (figures 2-3; column 4, lines 45-65); 

The cited portion of Szeliski (part of the Summary of the Invention) states: 

" The invention can be used to create full view panoramic 
mosaics from image sequences. Unlike current panoramic stitching 
methods, which usually require pure horizontal camera panning, the 
disclosed svstem does not require any controlled motions or constraints on 
how the images are taken (as long as there is no strong motion parallax). 
For example, images taken from a hand-held digital camera can be 
stitched seamlessly into panoramic mosaics. 

"By taking as many images as needed, image mosaics can 
be constructed which cover as large a field of view as desired, e.g., a 
complete environment map. Because the image mosaics is represented in 
the invention using a set of transforms, there are no singularity problems 
such as those existing at the top and bottom of cylindrical or spherical 
maps. This method is fast and robiast because it directly recovers 3D 
rotations instead of general 8 parameter planar perspective transforms. By 
mapping the mosaic onto an arbitrary texture-mapped polyhedron 
surrounding the origin, the virtual environment can be viewed or explored 
using standard 3D graphics viewers and hardware without requiring 
special-purpose players." (emphasis added't 

Figures 2A-3 are similar in scope. (See Szeliski, col. 7, lines 14-16 and col. 9, 

Unes 30-36) 

In addition to arguing the entire method of Szeliski in relation to 
step (a) of the method of Claim 1, the rejection also argues that parts of Szeliski 
disclose steps (b)-(d) of the method of Claim 1 , Taken together, the argument in 
the rejection requires support in Szeliski for performing the disclosed method, 
followed by repeating the parts of Szeliski cited in relation to steps (b)-(d) of the 
method of Claim 1. Absent that support, AppUcants request that the rejection be 
withdrawn. 

Szeliski also does not teach or suggest step (a) of Claim 1 . Claim 

1 requires: 

"(a) generating a plurality of three-dimensional panoramic 
images of a scene". 
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Szeliski discloses the preparation of panoramic mosaic images: 

"The focal length estimation method and apparatus claimed 
in this application align plural overlapping images with one another for 
constructing an image mosaic." (Szeliski, abstract) 
The panoramic mosaics of Szeliski lack depth information: 

"hnage-based rendering is a popular way to simulate a 
visually rich tele-presence or virtual reality experience. Instead of 
building and rendering a complete 3D model of the environment, a 
collection of images is used to render the scene while supporting virtual 
camera motion." (Szeliski, col. 1, lines 15-19) 

" The present invention is particularlv directed to image- 
based rendering systems without anv depth information , i.e., those which 
only support user panning, rotation, and zoom." (Szeliski, col. 1, lines 26- 
28; (emphasis added) 

"The specification discloses how to solve the problem of 
loss of detail or image ghosting is solved by computing local motion 
estimates (block-based optical flow) between pairs of overlapping images, 
and using these estimates to warp each input image so as to reduce the 
misregistration. This is less ambitious than actuallv recovering a 
perspective depth value for each pixel , but has the advantage of being able 
to simultaneously model other effects such as radial lens distortions and 
small movements in the image." (Szeliski, col. 4, lines 4-12) 
The panoramic mosaic images of Szeliski can be projected onto a three- 
dimensional surface for viewing: 

"The system can use a postprocessing stage to project such mosaics onto a 
convenient viewing surface, i.e., to create an environment map represented 
as a texture-mapped polyhedron svurounding the origin." (Szeliski, col. 3, 
lines 63-67; also note the 3-D model (polyhedron) in item 270 of Figure 
2B along with a 2-D texture map) 
The polyhedron is arbitrary, but allows use of a 3-D viewer. (Szeliski, col. 32, 
lines 34-37) The addition of arbitrary 3-D information (the polyhedron surface) 
does not tum a panoramic mosaic into a three-dimensional panoramic image. 

Szeliski teaches against step (b) of Claim 1. Claim 1 requires: 


"(b) determining transformations that align the plurality of 
three-dimensional panoramic images". 
The rejection relies upon Figures 4-5 and a section of Szeliski that has the 
heading: "Incremental 3-D Rotational Alignment". (Szeliski, Figures 4-5, and 
col. 13, line 25 to col. 14, line 45). In relation to incremental alignment, Szeliski 
does not teach aligning a plurality of three-dimensional panoramic images or even 
mosaic images, but rather aligning each 2-D image used to make a mosaic with 
the growing mosaic image or, for speed, with another 2-D image: 

"Incremental Alignment 

"In the invention, image mosedcs are represented as 
collections of images with associated geometric transformations. The first 
stage of the mosaic construction method computes an initial estimate for 
the transformation associated with each input image. This is done by 
processing each input image in turn, and finding the best alignment 
between this image and the mosaic constructed fi-om all previous images. 
(To speed up this part, one option is to register with only the previous 
image in the sequence,)" (Szeliski, col. 9, lines 8-17; also see discussion 
of Figure 4-5 at col. 9, lines 36-53) 

Szeliski's incremental alignment, 2-D image by 2-D image, is unlike Claim 1, 

which requires aligning a plurality of three-dimensional panoramic images, each 

derived firom a plurality of range images. 

Szeliski does not teach step (c) of Claim 1 . Claim 1 requires: 

"(c) integrating spatial information fi-om the plurality of 
three-dimensional panoramic images to form a spatial three-dimensional 
model of the scene". 

The rejection relies upon Figures 1 and 2B and states that 

"environment/texture map memory 270 stores a 3-D model". 

Szeliski does not teach formation of a spatial three-dimensional model. 

Summarizing the portions of Szeliski quoted above in relation to Claim 1 step (a): 

"Instead of building and rendering a complete 3D model of 
the environment, a collection of images is used to render the scene ..." 
(See Szeliski, col. 1, lines 15-19 at lines 17-19) 
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"The present invention is particularly directed to image- 
based rendering systems without any depth information" (Szeliski, col. 1, 
lines 26-27; see lines 26-28) 

"The specification discloses how to solve the problem of 
loss of detail or image ghosting ... This is less ambitious than actually 
recovering a perspective depth value for each pixel ..." (Szeliski, col. 4, 
lines 4-12 at lines 4-5 and 8-10) 
The 3-D model of item 270 is a general 3-D object onto which the mosaic image 
is mapped in postprocessing. (Szeliski, col. 32, lines 20-22; col. 3, lines 63-67) 
Szeliski states: 

"The shape of the model and the embedding of each face into texture 
space are left up to the user. This choice can range fi*om something as 
simple as a cube with six separate texture maps, to something as 
complicated as a subdivided dodecahedron, or even a latitude-longitude 
tessellated globe." (Szeliski, col. 27, lines 44-49) 
The model of Szeliski is not a model of the scene. The mosaic image after 
mapping is still a mosaic image. 

Szeliski does not teach step (d) of Claim 1. Claim 1 requires: 

"(d) integrating intensity and texture information fi-om the 
plurality of three-dimensional panoramic images onto the spatial three- 
dimensional model to form a three-dimensional model of the scene 
containing both spatial and intensity information." 
Szeliski does not integrate intensity and texture information from three 
dimensional panoramic images. Szeliski maps from a 2-D texture map: 

"FIGS. 27 and 28 illustrate a cubic 3D model and 
corresponding 2D texture map for carrying out one embodiment of a 
texture mapping method. 

"FIG. 29 illustrates a mapping between triangular vertices 
of a 3D model and a 2D texture map employed in carrying out one 
embodiment of the texture mapping. 

"FIGS. 30 and 31 illustrate a spherical 3D model and 
corresponding 2D texture map for carrying out one embodiment of the 
texture mapping method." (Szeliski, col. 6, lines 20-29) 
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As earlier discussed, there is also no 3-D model of the scene in Szeliski, just a 
mosaic image. 

The rejection states in relation to the combination of Szeliski and 

Huber: 

"Szeliski is silent about range images as claimed. Huber, in 
the same field of endeavor of automatic 3D modeling using range images 
obtained fi-om unknown viewpoints (title) and same problem solving field 
of range image information disclose the use of range images. In particular, 
Huber constructs a 3D model fi-om a plurality of range images using a 
surface matching algorithm. It would have been obvious to a person of 
ordinary skill in the art at the time of the invention was made to modify 
Szeliski by using range images in creating 3D panoramic image model as 
taught by Huber because such a modification will allow a system to create 
an automatically 3D model of a scene from a set of range images obtained 
from unknown viewpoints as mentioned Huber in the title and also at least 
in the abstract." 

The cited motivation supports use of Huber without Szeliski. As noted in the 
application: 

"Huberts algorithm for 3-D modeling generates a 3-D model from a series 
of range images, assuming nothing is known about the relative views of 
the object." (application, page 2, lines 17-19) 

Why would one of skill in the art not use the existing solution to problem: the 

method taught by Huber? 

Szeliski also teaches or suggests, that depth information could be 

provided by adding a depth map to a completed mosaic image: 

"Instead of building and rendering a complete 3D model of the 
environment, a collection of images is used to render the scene while 
supporting virtual camera motion. For example, a single cylindrical image 
surrounding the viewer enables the user to pan and zoom inside an 
environment created from real images. More powerfiil image-based 
rendering systems can be built by adding a depth map to the image or by 
using a larger collection of images." (Szeliski, col. 1, lines 17-25) 

Why would one of skill in the art ignore this teaching or suggestion of how to add 

depth (range) information and instead look to Huber? 
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Assuming for the sake of argument that Szeliski and Ruber could 
be combined, that combination would present the same shortcoming as Huber. 
The application describes this: 

"Ruber's algorithm for 3-D modeling generates a 3-D model from a series 
of range images, assuming nothing is known about the relative views of 
the object. It can be broken down into three phases: (1) determining 
which views contain overlaps, (2) determining the transformation between 
overlapping views, and (3) determining the global position of all views /' 
(application, page 2, lines 17-21: emphasis added) 

"Once the overlapping views and local transformations are 
estimated, step (3) of Ruber's algorithm entails using a series of 
consistency measures in combination with a model graph to find any 
inconsistencies in the local transformations. Huber recognizes, however, 
that there are computational costs in scaling his technique to a large 
number of views. For that reason, the computational cost of step (3) can 
grow prohibitivelv expensive as the number of input range images gets 
large ." (application, page 3, lines 13-19; emphasis added) 
Szeliski, as discussed above, uses a similar incremental, image-by-image 
alignment method. (Szeliski, col. 9, lines 8-17; also see discussion of Figure 4-5 
at col. 9, lines 36-53) The combination of Szeliski and Huber would still lack the 
advantage provided by the claimed invention: 

"The advantage of this invention is that the use of the 3-D 
panoramic images drastically simplifies the merging process compared to 
merging the entire set of individual range images. This invention enables 
a complete 3-D surface description to be easily derived for an arbitrary 
physical scene." (application, page 5, lines 10-14) 
In relation to this. Claim 1 states: 

"(b) determining transformations that align the plurality of 
three-dimensional panoramic images : 

"(c) integrating spatial information from the plurality of 
three-dimensional panoramic images to form a spatial three-dimensional 
model of the scene; and 

"(d) integrating intensity and texture information from the 
plurality of three-dimensional panoramic images ". 
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Claims 4 and 7-8 are allowable as depending from Claim 1 and on 
the same basis as Claim 1 . 

Claim 9 is allowable on the grounds discussed above in relation to 

Claim 1, 

Claims 10 and 13-14 are allowable as depending from Claim 9. 

Claims 5 and 1 1 stand rejected under 35 U.S.C. 103(a) as being 
impatentable over the combination of Sezliski and Huber as applied to claim 1, 
and fiirther in view of Nayar et al. (US 4,912,336). 

Claims 5 and 1 1 are allowable as depending from Claims 1 and 9, 

respectively. 

Claims 6 and 12 stand rejected under 35 U.S.C. 103(a) as being 
unpatentable over a combination of Szeliski and Huber as applied to claims 1 , 4, 
7-9 and 13-14 and further in view of applicant's admitted prior art in the 
specification at page 10, lines 4-14. 

Claims 6 and 12 are allowable as depending from Claims 1 and 9, 

respectively. 

Added Claim 15 is supported by the application as filed, notably 
the original claims and at page 7, lines 3-5 and 16-21 and is allowable on the 
same basis as Claim 1 . 

Added Claims 16-17 are supported on the same basis as Claim 15 
and are allowable as depending from Claim 1 5. 

Added Claim 18 is supported and allowable on the same grounds 

as Claim 15. 

Added Claims 19-20 are allowable as depending from Claim 18. 
Claim 1 9 is supported by the application as filed, notably original Claims 4 and 
10. Claim 20 is supported by the application as filed, notably at page 9, lines 8- 
12. 

It is believed that these changes now make the claims clear and 
definite and, if there are any problems with these changes. Applicants' attorney 
would appreciate a telephone call. 

In view of the foregoing, it is believed none of the references, 
taken singly or in combination, disclose the claimed invention. Accordingly, this 
application is believed to be in condition for allowance, the notice of which is 
respectfiiUy requested. 
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Respectfully submitted. 




Attorney for Applicant(s) 
Registration No. 30,700 

Robert Luke Walker/amb 
Rochester, NY 14650 
Telephone: (585) 588-2739 
Facsimile: (585) 477-1 148 

Enclosures: Replacement Drawings Figures 1-6 
Letter to the Draftsperson 
Copies of Formal Drawings 1-6 
Aimotated Drawing, Figure 6 

If the Examiner is unable to reach the AppUcant(s) Attorney at the telephone number provided, the 
Examiner is requested to communicate with Eastman Kodak Company Patent Operations at 
(585) 477-4656. 
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